Adding conversation context from detected audio to contact records

ABSTRACT

Provided are a computer program product, system, and method for adding conversation context from detected audio to contact records. An audio stream is received through a microphone of a personal computing device. Context analysis is performed of the audio stream to detect conversation context in the audio stream. A request is received to associate the conversation context with a selected contact person. A determination is made of conversation context detected from a segment of the audio stream received during a period of time from when the request was received. The conversation context determined for the period of time is included in a contact record for the selected contact person. The contact record is stored in a contact database including a plurality of contact records for a plurality of contact persons, wherein the contact records include conversation context and location information associated with the contact persons.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to a computer program product, system, andmethod for adding conversation context from detected audio to contactrecords.

2. Description of the Related Art

Current personal information managers include a contact database whereusers may maintain information on stored contacts, including name,address, email, phone and other contact information. The user may alsosupplement contact information with notes and other information, such asweb sites, pictures and attachments.

There is a need in the art for developing improved techniques forproviding enhanced information for contact records.

SUMMARY

Provided are a computer program product, system, and method for addingconversation context from detected audio to contact records. An audiostream is received through a microphone of a personal computing device.Context analysis is performed of the audio stream to detect conversationcontext in the audio stream. A request is received to associate theconversation context with a selected contact person. A determination ismade of conversation context detected from a segment of the audio streamreceived during a period of time from when the request was received. Theconversation context determined for the period of time is included in acontact record for the selected contact person. The contact record isstored in a contact database including a plurality of contact recordsfor a plurality of contact persons, wherein the contact records includeconversation context and location information associated with thecontact persons.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an embodiment of a personal computing device.

FIG. 2 illustrates an embodiment of a conversation context instance.

FIG. 3 illustrates an embodiment of a location information instance.

FIG. 4 illustrates an embodiment of a contact record.

FIG. 5 illustrates an embodiment of operations to collect conversationcontext and location information for an audio stream captured on amicrophone of a personal computing device.

FIGS. 6a and 6b illustrate an embodiment of operations to add collectedconversation context and location information to a selected new orexisting contact record.

FIGS. 7 and 8 illustrate embodiments of operations to search based onconversation context, location information, time periods, and contactsin the contact records.

DETAILED DESCRIPTION

Described embodiments provide improvements to computer technology toanalyze audio detected on a personal computing device and associateconversation context and location information for the detected audiostream with contact records in a contact database. Described embodimentsprovide improvements to the data structures and algorithms used togather and analyze audio information and associate with contact records.With described embodiments, the personal computing device maycontinually analyze a detected audio stream through a microphone of thecomputing device and store analyzed conversation context and locationinformation in a conversation context stream and location informationstream. This saved conversation context and location information forrecently held conversations may then be associated with data structuresfor personal contacts to be available for recall.

Described embodiments provide improved computer database technologies tosave recent conversation context and location information inconversation context stream and a location information stream datastructures. Further described embodiments provided technologicalimprovements to contact record data structures and contact databases toprovide an enhanced contact record to store gathered conversationcontext and location information for conversations held with thecontact.

FIG. 1 illustrates an embodiment of a personal computing device 100configured for use to assist a user in supplementing information oncontacts with information on previously held conversations andinteractions with the contact. The personal computing device 100includes a processor 102, a main memory 104, a communication transceiver106 to communicate (via wireless communication or a wired connection)with external devices, including a network, such as the Internet, acellular network, etc.; a microphone 110 to receive as input soundexternal to the personal computing device 100; a display screen 112 torender display output to a user of the personal computing device 100; aspeaker 114 to generate sound output to the user; input controls 116such as buttons and other software or mechanical buttons, including akeyboard, to receive user input; and a global positioning system (GPS)module 118 to determine a GPS potions of the personal computing device.The components 102-118 may communicate over one or more bus interfaces120.

The main memory 104 may include various program components including anaudio analyzer 122 to analyze context of audio content received over themicrophone 110, including a speech to text converter 124 to convertaudio and voice into written text to be analyzed, a context analyzer 128to analyze the context of the converted audio to determine conversationcontext, such as topics of conversation, names mentioned, locationsmentioned, emotions, etc. Conversation context instances gathered atdifferent times of the processed audio stream are stored in aconversation context stream 200. The memory 104 includes a historicalanalyzer 130 to determine whether user data sources, such as emails,texts, social media posts, etc., include names, locations, and topics ofconversation determined from the conversation context. The historicalanalyzer 130 may include application programming interfaces (APIs) toaccess user data sources, such as email, text accounts, social mediaaccounts, etc.

The memory 104 includes a geo-semantic analyzer 132 to processgeographical location information from the GPS module 118 to determineinformation on the geographical location, such as using a local or cloudbased mapping service to determine venue description of the geographicallocation, such as a name of the location, business, building, owner orentity name, etc., and then to access further information on thelocation, such as go to an Internet web site associated with the venuedetermined from the mapping service to determine events occurring at thevenue. The geo-semantic analyzer 132 may also access a user calendardatabase 134 having scheduled user calendar events to determine if thereis a scheduled calendar event for the time at which the geographicallocation was determined. The geo-semantic analyzer 132 may storegathered location information instances in a location information stream300 having location information instances for different times at whichthe geographical information was determined.

The memory 104 includes content analytics 136 to analyze different userdata sources, such as email accounts, messages, social media accounts,etc., and generate a content index 138 providing an indexed of theanalyzed data sources, such as posting lists. The historical analyzer130 may process the content index 138 to determine user content in userdata sources, either local or external to the personal computing device100, that is relevant to conversation context and location information.

The memory 104 includes a personal information manager 140 to managepersonal information of the user of the personal computing device 100,including the calendar database 134 having stored calendar events for anelectronic calendar and a contact database 400 having contact recordsproviding contact information for contacts. The personal informationmanager 140 may further interact with the audio analyzer 122, historicalanalyzer 130, geo-semantic analyzer 132, and the content analytics 136to manage and gather personal information for the user.

The speech to text convertor 124 may comprise a speech to text convertorprogram, such as, by way of example, the International Business MachinesCorporation (IBM) Watson™ Speech to Text service, the context analyzer128 may comprise a natural language processor, such as by way ofexample, the Watson™ Natural Language Understanding, providing APIs todetermine, from the text of the converted audio, topics of conversation,names, locations, and the Watson™ Tone Analyzer to analyze emotions andtones based on the text converted from audio. The content analytics 136may comprise a content manager or search crawler, such as the Watson™Content Analytics that collects and analyzes structured and unstructuredcontent. (IBM and Watson are trademarks of International BusinessMachines Corporation throughout the world).

The main memory 104 may further include an operating system 142 tomanage the personal computing device 100 operations and interface withdevice components 102-120.

The personal computing device 100 may comprise a smart phone, personaldigital assistance (PDA), or stationary computing device capable toprocess audio and voice detected by the microphone 110. The memory 104may comprise non-volatile and/or volatile memory types, such as a FlashMemory (NAND dies of flash memory cells), a non-volatile dual in-linememory module (NVDIMM), DIMM, Static Random Access Memory (SRAM),ferroelectric random-access memory (FeTRAM), Random Access Memory (RAM)drive, Dynamic RAM (DRAM), storage-class memory (SCM), Phase ChangeMemory (PCM), resistive random access memory (RRAM), spin transfertorque memory (STM-RAM), conductive bridging RAM (CBRAM), nanowire-basednon-volatile memory, magnetoresistive random-access memory (MRAM), andother electrically erasable programmable read only memory (EEPROM) typedevices, hard disk drives, removable memory/storage devices, etc.

The bus 120 represents one or more of any of several types of busstructures, including a memory bus or memory controller, a peripheralbus, an accelerated graphics port, and a processor or local bus usingany of a variety of bus architectures. By way of example, and notlimitation, such architectures include Industry Standard Architecture(ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA)bus, Video Electronics Standards Association (VESA) local bus, andPeripheral Component Interconnects (PCI) bus.

Generally, program modules, such as the program components 122, 124,128, 130, 132, 136, 140, 142, etc., may comprise routines, programs,objects, components, logic, data structures, and so on that performparticular tasks or implement particular abstract data types. Theprogram modules may be practiced in distributed cloud computingenvironments where tasks are performed by remote processing devices thatare linked through a communications network. In a distributed cloudcomputing environment, program modules may be located in both local andremote computer system storage media including memory storage devices.

The program components and hardware devices of the personal computingdevice 100 of FIG. 1 may be implemented in one or more computer systems,where if they are implemented in multiple computer systems, then thecomputer systems may communicate over a network.

The program components 122, 124, 128, 130, 132, 136, 140, 142 may beaccessed by the processor 102 from the memory 104 to execute.Alternatively, some or all of the program components 122, 124, 128, 130,132, 136, 140, 142 may be implemented in separate hardware devices, suchas Application Specific Integrated Circuit (ASIC) hardware devices.

The functions described as performed by the program components 122, 124,128, 130, 132, 136, 140, 142 may be implemented as program code in fewerprogram modules than shown or implemented as program code throughout agreater number of program modules than shown.

FIG. 2 illustrates an embodiment of a conversation context instance 200_(i), the context analyzer 128 inserts into the conversation contextstream 200, and includes conversation context 202, such as topics ofconversation, names, locations, classifications, external information,emotions, tone, etc., and a time 204 of an audio segment in a detectedaudio stream from which the conversation context 202 was determined.

FIG. 3 illustrates an embodiment of location information instance 300_(i), the geo-semantic analyzer 132 inserts into the locationinformation stream 300, and includes a geographical location 302, suchas from the GPS module 118; location attributes 304, such as a name ofthe location, events scheduled at the location as determined from anexternal website providing information for the location, such as anentity web site, or scheduled events from the calendar database 134; anda time 306 the geographical location 302 was determined.

FIG. 4 illustrates an embodiment of a contact record 400 _(i) in thecontact database 400 for a contact, including a contact recordidentifier (ID) 402; contact personal identifiers 404, such as anon-rigid designator of a person or entity, including a name,occupation, address, email, phone number, etc.; one or more conversationcontext instances 406 involving conversations in which the contact wasinvolved; one or more location information instances 408 providinginformation on a location of the personal computing device 100; and zeroor more user content instances 410 providing user content from contentsources, e.g., email, text messages, social media postings, found to berelevant to conversation context.

The data structures in FIGS. 2-4 show various fields of information. Inalternative embodiments, different fields of information may be providedthan that described, such as additional fields of information, orcertain described fields of information may not be included.

FIG. 5 illustrates an embodiment of operations performed by the audioanalyzer 122 and geo-semantic analyzer 132 to generate the conversationcontext stream 200 and location information stream 300 of informationavailable to include with contact records 400 _(i). Upon receiving (atblock 500) an audio stream through the microphone 110, which may beconverted to an audio file or other multimedia file having audio, thespeech to text 124 program converts (at block 502) the audio stream intotext. The context analyzer 128 accesses (at block 504) an audio segmentin the text and performs (at block 506) a context analysis of theaccessed audio segment in the audio stream to determine conversationcontext (e.g., topics, names, locations, emotions, etc.). The contextanalyzer 128 may also determine (at block 508) external information,e.g., headline news, that is contemporaneous with the receipt of theaudio, from an external source, such as a news web site over theInternet. The context analyzer 128 determines (at block 510) a time ofthe audio segment from which the conversations context is determined.The determined conversation context and determined external informationare saved (at block 512) as conversation context 202 and the determinedtime is saved as time 204 in a conversation context instance 200 _(i)stored in the conversation context stream 200.

At this point, or independently, the geo-semantic analyzer 132 maydetermine (at block 514) a geographical location, such as GPScoordinates, from the GPS module 118. If (at block 516) the geographicallocation changed from that in the previous generated locationinformation instance 300 _(i) in the location information stream 300,then the geo-semantic analyzer 132 determines (at block 518) from amapping service, which may be located in the personal computing device100 or at an external web site, a venue description for the geographicallocation, such as a name of a commercial or government establishmentdefined for the location, building, area, park, campus, etc. Thegeo-semantic analyzer 132 may then access (at block 520), from anInternet web site associated with venue, a description of an eventlisted on the web site for the venue at a time the geographical locationwas determined. For instance, if the location is a convention center,sports arena, etc., the web site for that venue may provide a calendarof scheduled events at the venue. The geo-semantic analyzer 132determines (at block 522) from the user calendar database 134 anycalendar events scheduled at the time the geographical location wasdetermined. A location information instance 300 _(i) is generated (atblock 524) indicating the geographical location in field 302, a time atwhich the geographical location was determined in field 306, andlocation attributes 304, such as the venue description for of thegeographical location, any events at the venue determined from anexternal web site over the Internet or calendar database 134, and otherdetermined location information.

If (at block 516) the geographical location has not changed or aftergenerating (at block 524) the location information, the context analyzer128 accesses (at block 526) a next audio segment in text form from theaudio stream and proceeds back to block 506 to process the next segmentfrom the audio stream.

The conversation context stream 200 and location information stream 300may comprise circular buffers to store a limited amount of information,such that when the buffers or stream have reached their storage limit,new instances 200 _(i) and 300 _(i) overwrite the oldest instances inthe stream 200 and 300, respectively. In this way, conversation contextand location information may be stored for a limited period of time,such as some hours, to be available to be associated with contactrecords 400.

With the operations of FIG. 5, conversation context and locationinformation for a user of the personal computing device 100 is generatedand stored in a conversation context stream 200 and location informationstream 300, respectively, to be available to associate with contactrecords if the user selects to include conversation context and locationinformation for a contact person that participated in the conversation.

FIGS. 6a and 6b provide an embodiment of operations performed by thepersonal information manager 140, historical analyzer 130, or some othercomponent to process a request by a user of the portable computingdevice 100 to associate conversation context and location informationwith a selected contact at a time, where the selected contact may be fora new contact, such as a person the user just spoke with or an existingcontact. Upon receiving (at block 600) the request, the personalinformation manager 140 determines (at block 602) whether the selectedcontact person is a new contact person to add to the contact database400 or an existing contact person for which there is an existing contactrecord 400 _(i) in the contact database 400. If (at block 602) theselected contact is an existing contact, then a determination is made(at block 604) of the contact record 400 _(i) for the selected existingcontact person in the contact database 400.

A determination is made (at block 606) of conversation context instances200 _(i) and location information instances 300 _(i) in the conversationcontext stream 200 and the location information stream 300 during aperiod of time from the time of the request.

The historical analyzer 130 may be called to determine from the contentindex 138 user content (e.g., emails, text messages, social media posts,etc.) from multiple local and remote data sources (email store,messages, posts on remote social media websites) including determinedconversation subject matter (topics, locations, names) and determinedlocation information (location name and attributes). The personalinformation manager 140 includes (at block 610) in the determinedcontact record 400, the determined conversation context instances 200_(i) in field 406, location information instances 300 _(i) in field 408,and determined user content in field 410, or links thereto.

If (at block 602) the selected contact person is a new contact to add tothe contact database 400, then the personal information manager 140determines (at block 612) whether the personal identifier or designatorof the new contact person (name, address, occupation, etc.) are relatedto personal identifiers 404 in the contact record 400 i of at least oneexisting contact person in the contact database 400 with a firstthreshold degree of certainty. If there is no existing contact personhaving related personal identifiers 404, then a new contact record 400_(N) is added (at block 614) to the contact database 400 including thenew contact personal identifiers in field 404, and control proceeds toblock 606 to add conversation context and location information to thenew contact record 400 _(N), or links thereto. If (at block 612) the newcontact personal identifiers match the personal identifiers 404 of oneor more existing contact records 400 _(i), for candidate existingcontact persons, then control proceeds to block 616 in FIG. 6b todetermine whether the conversation context and the location informationfor the period of time are related to a conversation context instance406 and a location information instance 408 in at least one existingcontact record 400 _(i) for at least one candidate existing contactperson with a second threshold degree of certainty.

If (at block 616) the information is related and if (at block 618) thereis only one candidate existing contact having related conversationcontext and location information, then the only on candidate existingcandidate contact person is selected (at block 620) as the selectedexisting contact person. If (at block 618) there are multiple candidateexisting contact persons with related conversation context and locationinformation, then the personal information manager 140 renders (at block622) on the display 112 a graphical user interface to enable the user ofthe personal computing device 100 to select one of the multiplecandidate contact persons with related conversation and locationinformation. Upon receiving (at block 624) user selection of one of themultiple candidate existing contact persons to use as the selectedexisting contact person or after selecting (at block 620) the only onecandidate, control proceeds (at block 626) to block 606 in FIG. 6a topopulate the contact records 400 _(i) for the selected existing contactperson(s) with the conversation context instance 200 _(i) and locationinformation instances 300 _(i) created from conversation audio during arecent time period from when the request was made.

If (at block 616) there is no relation of the determined conversationcontext and location information in any of the contact records forcandidate existing contact persons having matching personal identifiers,then control proceeds (at block 628) to block 612 in FIG. 6a to add anew contact record 400 _(N).

With the operations of FIGS. 6a and 6b , the user may decide when to addconversation context and location information for recent conversationsand current locations to a new or existing contact record. In this way,the user may supplement contact records with conversation context ofprevious meetings with the contact to be able to recall to enhancefuture conversations with the contact at a later point in time.

FIG. 7 illustrates an embodiment of operations performed by the personalinformation manager 140 to process a search request providingconversation context and a time period to search to determine contactsinvolved in conversations having the searched conversation context. Uponreceiving (at block 700) a search request including at least oneconversation context search term, such as a topic of conversation, and atime period search term, the personal information manager 140 processes(at block 702) an index or the contact records 400 _(i) to determine anycontact records 400 _(i) having conversation context instances 406and/or location information instances 408 related to the at least oneconversation context and location search terms and associated with atime period matching the time period search term. If (at block 704)there are contact records 400 _(i) satisfying the search criteria, thenthe personal information manager 140 returns (at block 706) informationon each contact person associated with a contact record 400, satisfyingthe search terms, i.e., involved in a conversation or location duringthe search time period.

With the search of FIG. 7, if a user recalls a topic of conversations ordetails of a location of an event during a time period, but does notremember the people involved in such conversations, then the searchcriteria of FIG. 7 may be used to determine those contacts involved inconversations involving the searched conversation topic or at thelocation matching the location search term.

FIG. 8 illustrates an embodiment of operations performed by the personalinformation manager 140 to process a search request providing a personalidentifier search term and time period search term to determine theconversation context and location information for the searched contactat the time period. Upon receiving (at block 800) the search request,the personal information manager 140 determines (at block 802) if thereis a contact record 400 _(i) for a contact person having a personalidentifier 404 related to the personal identifier search term. If (atblock 802) a matching contact person is found, the personal informationmanager 140 determines (at block 804) whether the contact record 400_(i) satisfying the personal identifier search term includes at leastone of conversation context instance 406 and location informationinstance 408 having times 204, 306 satisfying the time period searchterm. If (at block 804) there the matching contact record providesconversation context and location information for the search timeperiod, then is a match, then the personal information manager 140returns (at block 806) information for the contact person having thepersonal identifier related to the personal identifier search term andthe any determined conversation context instance 406 and/or locationinformation instance 408 having a time 204 and 406 satisfying the timeperiod search term.

With the search of FIG. 8, a user may know of a contact name and seek todetermine whether there were any conversations with that contact for aspecified time period to return. Obtaining information on conversationcontext and location information where the user met the contact at thespecified time period allows the user to recall pertinent issues todiscuss with the contact or enhance and expand the topics ofconversation with the contact when later engaged after a previousconversation.

In further embodiments, other searches than those described with respectto FIGS. 7 and 8 may be performed, such as searches just one locationinformation or conversation context, and different combinations of thepossible search terms for personal identifiers, conversation context,location information, and time periods.

The determination of whether information is related may use data miningalgorithms known in the art to determine records having matching orrelated information that are related with a degree of certainty. Thefirst and second threshold degrees of certainty used for differentdeterminations may be the same or different degrees of certainty.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Java, Smalltalk, C++ or the like,and conventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The letter designators, such as i and n, used to designate a number ofinstances of an element may indicate a variable number of instances ofthat element when used with the same or different elements.

The terms “an embodiment”, “embodiment”, “embodiments”, “theembodiment”, “the embodiments”, “one or more embodiments”, “someembodiments”, and “one embodiment” mean “one or more (but not all)embodiments of the present invention(s)” unless expressly specifiedotherwise.

The terms “including”, “comprising”, “having” and variations thereofmean “including but not limited to”, unless expressly specifiedotherwise.

The enumerated listing of items does not imply that any or all of theitems are mutually exclusive, unless expressly specified otherwise.

The terms “a”, “an” and “the” mean “one or more”, unless expresslyspecified otherwise.

Devices that are in communication with each other need not be incontinuous communication with each other, unless expressly specifiedotherwise. In addition, devices that are in communication with eachother may communicate directly or indirectly through one or moreintermediaries.

A description of an embodiment with several components in communicationwith each other does not imply that all such components are required. Onthe contrary a variety of optional components are described toillustrate the wide variety of possible embodiments of the presentinvention.

When a single device or article is described herein, it will be readilyapparent that more than one device/article (whether or not theycooperate) may be used in place of a single device/article. Similarly,where more than one device or article is described herein (whether ornot they cooperate), it will be readily apparent that a singledevice/article may be used in place of the more than one device orarticle or a different number of devices/articles may be used instead ofthe shown number of devices or programs. The functionality and/or thefeatures of a device may be alternatively embodied by one or more otherdevices which are not explicitly described as having suchfunctionality/features. Thus, other embodiments of the present inventionneed not include the device itself.

The foregoing description of various embodiments of the invention hasbeen presented for the purposes of illustration and description. It isnot intended to be exhaustive or to limit the invention to the preciseform disclosed. Many modifications and variations are possible in lightof the above teaching. It is intended that the scope of the invention belimited not by this detailed description, but rather by the claimsappended hereto. The above specification, examples and data provide acomplete description of the manufacture and use of the composition ofthe invention. Since many embodiments of the invention can be madewithout departing from the spirit and scope of the invention, theinvention resides in the claims herein after appended.

1. A computer program product for providing contact information forcontacts in a personal computing device in communication with amicrophone, the computer program product comprising a computer readablestorage medium having computer readable program code embodied thereinthat executes to perform operations, the operations comprising:receiving an audio stream through the microphone; performing contextanalysis of the audio stream to detect conversation context in the audiostream; receiving a request to associate the conversation context with aselected contact person; determining conversation context detected froma segment of the audio stream received during a period of time from whenthe request was received; including in a contact record for the selectedcontact person the conversation context determined for the period oftime; and storing the contact record in a contact database including aplurality of contact records for a plurality of contact persons, whereinthe contact records include conversation context and locationinformation associated with the contact persons.
 2. The computer programproduct of claim 1, wherein the conversation context comprises at leastone of topics of conversation, names, locations, and emotions determinedby performing context analysis of the audio stream, wherein theoperations further comprise: determining user content from multiple datasources related to the topics of conversation, names and locationsdetected from the segment of the audio stream received during the periodof time from when the request was received; and including in the contactrecord , for the selected contact person, information on the usercontent.
 3. The computer program product of claim 1, wherein theselected contact person comprises a new contact person to add to thecontact database, wherein the operations further comprise: determiningwhether a personal identifier of the new contact person is related to apersonal identifier of an existing contact person having an existingcontact record in the contact database with a threshold degree ofcertainty, wherein the contact record is created for the new contactperson in response to determining that the personal identifier of thenew contact person is not related to the personal identifier of anexisting contact person in the contact database; and including in theexisting contact record for the existing contact person the conversationcontext determined for the period of time.
 4. The computer programproduct of claim 3, wherein the threshold degree of certainty comprisesa first threshold degree of certainty, wherein the determining whetherthe personal identifier of the new contact person is related to thepersonal identifier of the existing contact person with the thresholddegree of certainty comprises: determining at least one candidateexisting contact persons having personal identifiers related to thepersonal identifiers of the new contact person with the first thresholddegree of certainty; for each existing contact record for each the atleast one candidate existing contact person, determining whether theconversation context for the period of time are related to theconversation context in the existing contact record with a secondthreshold degree of certainty; and selecting the candidate existingcontact person to merge with the new contact person comprising an onlyone of the at least one candidate existing contact person whose existingcontact record includes the conversation context related to theconversation context for the period of time with the second thresholddegree of certainty, wherein the first and the second threshold degreesof certainty may comprise a same and/or different degrees of certainty.5. The computer program product of claim 4, wherein the operationsfurther comprise: rendering on a display of the personal computingdevice a graphical user interface to enable a user of the personalcomputing device to select one of multiple candidate contact persons touse for the new contact person in response to determining that there aremultiple candidate contact persons having existing contact records withconversation context related to the conversation context for the periodof time with the second threshold degree of certainty; and receivingthrough the graphical user interface user selection of one of themultiple candidate contact persons rendered on the graphical userinterface, wherein the conversation context and the location informationdetermined for the period of time are added to the existing contactrecord for a selected candidate contact person.
 6. The computer programproduct of claim 1, wherein time information is provided with the audiostream indicating time information for audio segments in the audiostream, wherein the operations further comprise: saving, in aconversation context stream, conversation context instances resultingfrom the context analysis of the audio segments the audio stream and,for each conversation context instance, a time of an audio segment fromwhich the context conversation context was determined, wherein theconversation context instances have times within the period of time. 7.The computer program product of claim 1, wherein the operations furthercomprise: determining from a geographical location of the personalcomputing device location information including a description of a venueof the geographical location; determining location informationdetermined for at least one geographical location of the personalcomputing device during the period of time from when the request wasreceived; and including in the contact record for the selected contactperson the location information determined for the period of time. 8.The computer program product of claim 7, wherein the operations furthercomprise: receiving a search request including at least one of aconversation context search term and location search term and a timeperiod search term; and determining at least one contact record of thecontact records in the contact database including conversation contextand location information related to the at least one of the conversationcontext search term and the location search term and associated with atime period matching the time period search term; and returninginformation on each contact person associated with the determined atleast one contact record including the at least one of the conversationcontext search term and the location search term.
 9. The computerprogram product of claim 7, wherein the conversation context and thelocation information included in the contact record are associated witha time period when the request was received, wherein contact records fora plurality of contact persons in the contact database includeconversation context and location information for multiple time periods.10. The computer program product of claim 9, wherein the operationsfurther comprise: receiving a search request including a personalidentifier search term and a time period search term; determining acontact record for a contact person having a personal identifier relatedto the personal identifier search term; determining from the determinedcontact record at least one of the conversation context and the locationinformation associated with a time period matching the time periodsearch term; and returning information for the contact person having thepersonal identifier related to the personal identifier search term andthe at least of the conversation context and the location informationassociated with the time period matching the time period search term.11. The computer program product of claim 7, wherein the determining thelocation information further comprises: searching a calendar of a userof the personal computing device for any scheduled events during theperiod of time; and indicating the scheduled events in the contactrecord for the selected contact person.
 12. The computer program productof claim 7, wherein the determining the location information for thegeographical location further comprises: determining a description of avenue for the geographical location through a mapping service;accessing, from over a network, a description of an event at the venueand that occurred during the period of time; and including thedescription of the event in the contact record for the selected contactperson.
 13. The computer program product of claim 7, wherein theoperations further comprise: saving, in a location information stream,location information instances determined from geographical locations ofthe personal computing device and, for each location informationinstance, a time at which the location information was determined,wherein the location information comprises at least one locationinformation instance having a time within the period of time.
 14. Asystem for providing contact information for contacts, comprising: aprocessor; a microphone; a computer readable storage medium havingcomputer readable program code that when executed by the processorperform operations, the operations comprising: receiving an audio streamthrough the microphone; performing context analysis of the audio streamto detect conversation context in the audio stream; receiving a requestto associate the conversation context with a selected contact person;determining conversation context detected from a segment of the audiostream received during a period of time from when the request wasreceived; including in a contact record for the selected contact personthe conversation context determined for the period of time; and storingthe contact record in a contact database including a plurality ofcontact records for a plurality of contact persons, wherein the contactrecords include conversation context and location information associatedwith the contact persons.
 15. The system of claim 14, wherein theconversation context comprises at least one of topics of conversation,names, locations, and emotions determined by performing context analysisof the audio stream, wherein the operations further comprise:determining user content from multiple data sources related to thetopics of conversation, names and locations detected from the segment ofthe audio stream received during the period of time from when therequest was received; and including in the contact record , for theselected contact person, information on the user content.
 16. The systemof claim 14, wherein the selected contact person comprises a new contactperson to add to the contact database, wherein the operations furthercomprise: determining whether a personal identifier of the new contactperson is related to a personal identifier of an existing contact personhaving an existing contact record in the contact database with athreshold degree of certainty, wherein the contact record is created forthe new contact person in response to determining that the personalidentifier of the new contact person is not related to the personalidentifier of an existing contact person in the contact database; andincluding in the existing contact record for the existing contact personthe conversation context determined for the period of time.
 17. Thesystem of claim 16, wherein the threshold degree of certainty comprisesa first threshold degree of certainty, wherein the determining whetherthe personal identifier of the new contact person is related to thepersonal identifier of the existing contact person with the thresholddegree of certainty comprises: determining at least one candidateexisting contact persons having personal identifiers related to thepersonal identifiers of the new contact person with the first thresholddegree of certainty; for each existing contact record for each the atleast one candidate existing contact person, determining whether theconversation context for the period of time are related to theconversation context in the existing contact record with a secondthreshold degree of certainty; and selecting the candidate existingcontact person to merge with the new contact person comprising an onlyone of the at least one candidate existing contact person whose existingcontact record includes the conversation context related to theconversation context for the period of time with the second thresholddegree of certainty, wherein the first and the second threshold degreesof certainty may comprise a same and/or different degrees of certainty.18-20. (canceled)
 21. The system of claim 17, wherein the operationsfurther comprise: rendering on a display of the personal computingdevice a graphical user interface to enable a user of the personalcomputing device to select one of multiple candidate contact persons touse for the new contact person in response to determining that there aremultiple candidate contact persons having existing contact records withconversation context related to the conversation context for the periodof time with the second threshold degree of certainty; and receivingthrough the graphical user interface user selection of one of themultiple candidate contact persons rendered on the graphical userinterface, wherein the conversation context and the location informationdetermined for the period of time are added to the existing contactrecord for a selected candidate contact person.
 22. The system of claim14, wherein time information is provided with the audio streamindicating time information for audio segments in the audio stream,wherein the operations further comprise: saving, in a conversationcontext stream, conversation context instances resulting from thecontext analysis of the audio segments the audio stream and, for eachconversation context instance, a time of an audio segment from which thecontext conversation context was determined, wherein the conversationcontext instances have times within the period of time.
 23. The systemof claim 14, wherein the operations further comprise: determining from ageographical location of the personal computing device locationinformation including a description of a venue of the geographicallocation; determining location information determined for at least onegeographical location of the personal computing device during the periodof time from when the request was received; and including in the contactrecord for the selected contact person the location informationdetermined for the period of time.
 24. The system of claim 23, whereinthe operations further comprise: receiving a search request including atleast one of a conversation context search term and location search termand a time period search term; and determining at least one contactrecord of the contact records in the contact database includingconversation context and location information related to the at leastone of the conversation context search term and the location search termand associated with a time period matching the time period search term;and returning information on each contact person associated with thedetermined at least one contact record including the at least one of theconversation context search term and the location search term.
 25. Thesystem of claim 23, wherein the conversation context and the locationinformation included in the contact record are associated with a timeperiod when the request was received, wherein contact records for aplurality of contact persons in the contact database includeconversation context and location information for multiple time periods.26. The system of claim 25, wherein the operations further comprise:receiving a search request including a personal identifier search termand a time period search term; determining a contact record for acontact person having a personal identifier related to the personalidentifier search term; determining from the determined contact recordat least one of the conversation context and the location informationassociated with a time period matching the time period search term; andreturning information for the contact person having the personalidentifier related to the personal identifier search term and the atleast of the conversation context and the location informationassociated with the time period matching the time period search term.27. The system of claim 23, wherein the determining the locationinformation further comprises: searching a calendar of a user of thepersonal computing device for any scheduled events during the period oftime; and indicating the scheduled events in the contact record for theselected contact person.
 28. The system of claim 23, wherein thedetermining the location information for the geographical locationfurther comprises: determining a description of a venue for thegeographical location through a mapping service; accessing, from over anetwork, a description of an event at the venue and that occurred duringthe period of time; and including the description of the event in thecontact record for the selected contact person.
 29. The system of claim23, wherein the operations further comprise: saving, in a locationinformation stream, location information instances determined fromgeographical locations of the personal computing device and, for eachlocation information instance, a time at which the location informationwas determined, wherein the location information comprises at least onelocation information instance having a time within the period of time.